We will add an explicit comment on that.

(line 45).

"Is Approval V oting Optimal Given Approval V otes?" NeurIPS-2015. The reviewer points to a paper about "threshold voting," but this is a very different idea.

We thank all reviewers for their valuable comments. We'll further improve in the final version. Q1: Beyond regression tasks: In this paper, we focus on regression tasks. It's indeed an interesting topic to systematically investigate on the robustness of We note that there is some recent work (e.g., [*1]) that studies the robustness of the MMD estimators, Q3. Results of larger networks: Guo et al. (ICML 2017) argued that the miscalibration was due to the sheer size of We'll make it clearer in the final version.

The paper considers a new differentially private learning setting, that receives a collection of unlabeled public data, on top of the labelled private data of interests and assumes that the two data sets are drawn from the same distribution. The proposed technique allows the use of (non-private) agnostic PAC learners as black boxes oracles, which, when combining with and adapts to the structure of the data sets. The idea is summarized below: 1. Do differentially private model-serving in a data-adaptive fashion, through sparse vector'' technique and subsample-and-aggregate''. This only handles a finite number of classification queries. It behaves similarly to 1 using the properties of an agnostic PAC learner, but can now handle an unbounded number of classification queries.